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DETAILED ACTION 
Election/Restrictions 

Restriction to one of the following inventions is required under 35 U.S.C. 121: 

I. Claims 1-13 and 19-20 are drawn to a method and computer-readable 
medium comprising representing an element with shape on visual design 
surface, determining a value of at least one characteristic of the element, 
associating said characteristic with an aura and displaying the aura in 
association with shape on design surface, classified in class 345 subclass 
581 

II. Claims 14-18 are drawn to a computer system of changing at least one 
characteristic of an element displayed on a visual design surface, 
comprising displaying at least two affinity regions, wherein each affinity 
region includes elements having same value of a characteristic, receiving 
a movement command indicative of user interface selection device 
dragging an element from first affinity region and hovering the element 
over a second affinity region and in response changing a value of the 
characteristic of the element from the value associated with first affinity 
region to value associated with second affinity region, classified in class 
345 subclass 672 

The inventions are distinct, each from the other because of the following reasons: 
Inventions I and II are related as subcombinations disclosed as usable together 
in a single combination. The subcombinations are distinct if they do not overlap in 
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scope and are not obvious variants, and if it is shown that at least one subcombination 
is separately usable. In the instant case, invention I has separate utility such as 
associating at least one characteristic with an aura and displaying the aura in 
association with shape on the design surface. Invention II has separate utility such as 
receiving a movement command indicative of user interface selection device dragging 
an element from first affinity region and hovering the element over a second affinity 
region and changing a value of the characteristic of element from value associated with 
first affinity region to value associated with second affinity region. See MPEP 
§ 806.05(d). 

Because these inventions are independent or distinct for the reasons given 
above and there would be a serious burden on the examiner if restriction is not required 
because the inventions require a different field of search (see MPEP § 808.02), 
restriction for examination purposes as indicated is proper. 

The examiner has required restriction between subcombinations usable together. 
Where applicant elects a subcombination and claims thereto are subsequently found 
allowable, any claim(s) depending from or otherwise requiring all the limitations of the 
allowable subcombination will be examined for patentability in accordance with 37 CFR 
1.104. See MPEP § 821.04(a). Applicant is advised that if any claim presented in a 
continuation or divisional application is anticipated by, or includes all the limitations of, a 
claim that is allowable in the present application, such claim may be subject to 
provisional statutory and/or nonstatutory double patenting rejections over the claims of 
the instant application. 
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During a telephone conversation with Charles Miller on 2/26/07 a provisional 
election was made without traverse to prosecute the invention of a method and 
computer-readable medium comprising representing an element with shape on visual 
design surface, determining a value of at least one characteristic of the element, 
associating said characteristic with an aura and displaying the aura in association with 
shape on design surface, claims 1-13 and 19-20. Affirmation of this election must be 
made by applicant in replying to this Office action. Claims 14-18 are withdrawn from 
further consideration by the examiner, 37 CFR 1142(b), as being drawn to a.non- 
elected invention. 

Specification 

The disclosure is objected to because of the following informalities: p. 5 lines 21- 
26 seem to refer to the visual design surfaces and design elements of Figure 3 but the 
disclosure associates said surfaces and elements with Figure 2. Thus, appropriate 
correction is required. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-4, 8-13, 19-20 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Luo (20030198386) in view of Bates (6976214) 
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Regarding claim 1 , Luo teaches representing an element with a shape on a 
visual design surface. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 
paragraphs 37-38, paragraph 43, p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) It 
should be noted that the Luo teaches a character, which represents an element with a 
shape. (See. Fig. 5, Fig. 8B for example) Furthermore Luo teaches determining a value 
of at least one characteristic of the element. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 
paragraph 37, p. 4 paragraphs 37-38, paragraph 43, p. 5 paragraphs 45-48, p. 6 
paragraphs 49-50) It should be noted that Luo teaches for each character in the 
character string extracted, edge detection is utilized to determine whether a candidate 
box corresponds to a positive candidate (correspond to text regions) where edges are 
detected. Furthermore Luo teaches associating the at least one characteristic with an 
aura. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 paragraphs 37-38, 
paragraph 43, p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that the 
aura as taught by the Luo is bounding box bounding the area of the character and 
wherein each bounding box is associated with edge detection values as well as 
foreground/background contrast values, (p. 4 paragraphs 37-38, paragraph 43, p. 5 
paragraphs 45-48, p. 6 paragraphs 49-50) Nonetheless, Luo does not explicitly teach 
displaying the aura in association with the shape on the design surface. This is what 
Bates teaches. (Figs. 4-6, Col 4 line 4-Col 5 line 56) It should be noted that Bates 
teaches displaying text box in bold and highlighting the term in the text box. It would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to combine the teachings of displaying a term (word) inside a text box in bold and/or 
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highlighting the term in the text box because the user is provided with the additional 
functionality editing text and thus, improving composition and form of text. 

Regarding claim 19, Luo teaches a design element shape and an aura in 
proximity to the design element shape to represent a value of a characteristic. (Fig. 2, 
Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 paragraphs 37-38, paragraph 
43, p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that the aura as 
taught by the Luo is bounding box bounding the area of the character and wherein each 
bounding box is associated with edge detection values as well as foreground, 
background contrast values, (p. 4 paragraphs 37-38, paragraph 43, p. 5 paragraphs 45- 
48, p. 6 paragraphs 49-50) Nonetheless, Luo does not explicitly teach displaying the 
design shape element (character) and displaying the aura. This is what Bates teaches. 
(Figs. 4-6, Col 4 line 4-Col 5 line 56) It should be noted that Bates teaches displaying 
text box in bold and highlighting the term in the text box. It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to combine the 
teachings of displaying a term (word) inside a text box in bold and/or highlighting the 
term in the text box because the user is provided with the additional functionality editing 
text and thus, improving composition and form of text. 

Regarding claim 20, Luo teaches the aura is around the design element shape. 
(Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 paragraphs 37-38, 
paragraph 43, p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that the 
aura as taught by the Luo is bounding box bounding the area of the character and 
wherein each bounding box is associated with edge detection values as well as 
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foreground, background contrast values, (p. 4 paragraphs 37-38, paragraph 43, p. 5 
paragraphs 45-48, p. 6 paragraphs 49-50) Nonetheless, Luo does not explicitly teach 
displaying the aura. This is what Bates teaches. (Figs. 4-6, Col 4 line 4-Col 5 line 56) It 
should be noted that Bates teaches displaying text box in bold and highlighting the term 
in the text box. It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to combine the teachings of displaying a term (word) inside a 
text box in bold and/or highlighting the term in the text box because the user is provided 
with the additional functionality editing text and thus, improving composition and form of 
text.* 

Regarding claim 2, Luo teaches the aura comprises a color coded area 
•surrounding the shape. It should be noted that the image extracted is a gray scale 
image and the grayscale image is used to obtain edge representation of the image, (p. 2 
paragraph 27, p. 4 paragraph 36) 

Regarding claim 3, Luo teaches the aura comprises a color-coded area adjacent 
to at least a portion of the shape, (p. 2 paragraph 27, p. 4 paragraph 36, Fig. 2, Figs. 
8A-8B) 

Consider claim 4, Luo teaches repeating steps in claim 1 for a plurality of 
elements (i.e. additional characters of the extracted word, each character contained in a 
bounding box) (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 
paragraphs 37-38, paragraph 43, p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) 
Furthermore, Luo teaches merging aura associated with at least some elements in 
close proximity and having the same value characteristic. (Figs. 8A-8B, p. 5 paragraphs 
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45-48, p. 6 paragraphs 49-50) It should be noted that the common value characteristic 
as taught by Luo for each element (character) is acceptable value of contrast between 
foreground and background, (p. 5 paragraph 46) Nonetheless, Luo does not explicitly 
teach displaying the aura in association with the shape on the design surface. This is 
what Bates teaches. (Figs. 4-6, Col 4 line 4-Col 5 line 56) It should be noted that Bates 
teaches displaying text box in bold and highlighting the term in the text box. It would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to combine the teachings of displaying a term (word) inside a text box in bold and/or 
highlighting the term in the text box because the user is provided with the additional 
functionality editing text and thus, improving composition and form of text. 

Regarding claim 8, Luo teaches the at least one characteristic comprises a use 
for the element, (p.4 paragraph 37, p. 4 paragraphs 37-38, paragraph 43, p. 5 
paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that the edge 
(characteristic) allows the character to be used (detected) within a word. 

Regarding claim 11, Luo teaches wherein at least one characteristic comprises 
an identification of an importance level. (p,4 paragraph 37, p. 4 paragraphs 37-38, 
paragraph 43, p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that 
text may be recognized as of higher importance level than non-text. 

Consider claim 12, Luo teaches determination a value of at least as a second 
characteristic of the element. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, 
p. 4 paragraphs 37-38, paragraph 43, p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) It 
should be noted that a second characteristic as taught by Luo is contrast between 
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foreground and background, (p. 5 paragraphs 45-46) Associating the second 
characteristic with a second aura is taught in Fig. 8B (Merging auras to create a word). 
Nonetheless, Luo does not explicitly teach displaying the second aura in association 
with the shape on the design surface. This is what Bates teaches. (Figs. 4-6, Col 4 line 
4-Col 5 line 56) It should be noted that Bates teaches displaying text box in bold and 
highlighting the term in the text box. It would have been obvious to one of ordinary skill 
in the art at the time the invention was made to combine the teachings of displaying a 
term (word) inside a text box in bold and/or highlighting the term in the text box because 
the user is provided with the additional functionality editing text and thus, improving 
composition and form of text. 

Claim 13 is similar in scope to claim 12 except for the recitation of determining a 
value of at least a third characteristic of the element. This is taught by Luo. (p.4 
paragraphs 39-42) It should be noted that the third characteristic as taught by Luo is 
direction information. Thus, claim 13 is rejected under similar rationale as claim 12. 

Regarding claim 9, Luo does not explicitly teach one characteristic comprises 
identification of namespace. Examiner takes official notice that extracted words may be 
names. It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to extract names into the combination of Luo and Bates because 
the user is provided with the additional functionality of editing names. 

Regarding claim 10, Luo does not explicitly teach one characteristic comprises 
identification of an application layer. Examiner takes official notice that images with 
some text may identify an application layer. It would have been obvious to one of 
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ordinary skill in the art at the time the invention was made to provide highlights to those 
type of images to better visually identify and label said images. 

Claims 5 are rejected under 35 U.S.C. 103(a) as being unpatentable over Luo 
(200301 98386) in view of Bates (6976214) in further view of Tappert. (4731 857) 

Regarding claim 5, Luo teaches repeating steps in claim 1 for a plurality of 
elements (i.e. additional characters of the extracted word, each character contained in a 
bounding box) (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 
paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) For 
motivation of displaying the aura in association with shape, see claim 1. Nonetheless 
neither Luo nor Bates explicitly teaches sorting the elements (characters) so that 
elements having the same characteristics are generally in the same location. However, 
it should be noted that Luo does teach elements having the same characteristic 
(positive candidate bounding boxes) are generally located in the same region (see Fig. 
8B, merging of bounding boxes [auras]). However Luo fails to teach sorting of 
characters. This is what Bates teaches. (Col 3 lines 7-19, Col 3 lines 48-68, Col 5 lines 
1-18, Col 6line 51 -Col 8 line 29) It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to sort characters for word recognition 
purposes into the system of Luo because initial character segmentation is not final and 
need not be highly accurate but is subject to lesser constraint of containing all the true 
segmentation points (Col 3 lines 64-68) and thus, a more efficient word recognition 
system can be achieved. 



Application/Control Number: 10/798,206 Page 11 

Art Unit: 2628 

Regarding claim 6, Luo teaches grouping elements in affinity regions, wherein 
each affinity region represents elements having the same value of the characteristic. 
(Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 paragraphs 37-38, 
paragraph 43 r p. 5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that 
Luo teaches mergining two auras both having positive candidates (edge detected 
and/or proper allowed constrast levels). For motivation of displaying elements in 
merged auras (affinity regions), see claim 1. 

Regarding claim 7, Examiner takes official notice that it is well known to utilize 
labels such an image to indicate the status of an object or its contents. It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
utilize a label to allow user to identify and change the status of a highlighted text box 
into the system of Luo because labels provide the functionality of giving users a 
convenient way of identification as well as describe characteristics of the object that 
would not be obvious from simple examination. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kevin K. Xu whose telephone number is 571-272-7747, 
The examiner can normally be reached on 8:30AM - 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark Zimmerman can be reached on 571-272-7653. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 



Application/Control Number: 10/798,206 



Page 12 



Art Unit: 2628. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 



For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Status information for unpublished applications is available through Private PAIR only. 




Kevin Xu 



2/26/07 




MARK ZIMMERMAN 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



